{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout\n",
    "Dropout [1] is a technique for regularizing neural networks by randomly setting some features to zero during the forward pass. In this exercise you will implement a dropout layer and modify your fully-connected network to optionally use dropout.\n",
    "\n",
    "[1] Geoffrey E. Hinton et al, \"Improving neural networks by preventing co-adaptation of feature detectors\", arXiv 2012"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# As usual, a bit of setup\n",
    "\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from cs231n.classifiers.fc_net import *\n",
    "from cs231n.data_utils import get_CIFAR10_data\n",
    "from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array\n",
    "from cs231n.solver import Solver\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "def rel_error(x, y):\n",
    "  \"\"\" returns relative error \"\"\"\n",
    "  return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_val:  (1000, 3, 32, 32)\n",
      "X_train:  (49000, 3, 32, 32)\n",
      "X_test:  (1000, 3, 32, 32)\n",
      "y_val:  (1000,)\n",
      "y_train:  (49000,)\n",
      "y_test:  (1000,)\n"
     ]
    }
   ],
   "source": [
    "# Load the (preprocessed) CIFAR10 data.\n",
    "\n",
    "data = get_CIFAR10_data()\n",
    "for k, v in data.iteritems():\n",
    "  print '%s: ' % k, v.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout forward pass\n",
    "In the file `cs231n/layers.py`, implement the forward pass for dropout. Since dropout behaves differently during training and testing, make sure to implement the operation for both modes.\n",
    "\n",
    "Once you have done so, run the cell below to test your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running tests with p =  0.3\n",
      "Mean of input:  9.99986926314\n",
      "Mean of train-time output:  9.99423641269\n",
      "Mean of test-time output:  9.99986926314\n",
      "Fraction of train-time output set to zero:  0.300364\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.6\n",
      "Mean of input:  9.99986926314\n",
      "Mean of train-time output:  10.0071595227\n",
      "Mean of test-time output:  9.99986926314\n",
      "Fraction of train-time output set to zero:  0.599892\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.75\n",
      "Mean of input:  9.99986926314\n",
      "Mean of train-time output:  10.0047696473\n",
      "Mean of test-time output:  9.99986926314\n",
      "Fraction of train-time output set to zero:  0.749912\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(500, 500) + 10\n",
    "\n",
    "for p in [0.3, 0.6, 0.75]:\n",
    "  out, _ = dropout_forward(x, {'mode': 'train', 'p': p})\n",
    "  out_test, _ = dropout_forward(x, {'mode': 'test', 'p': p})\n",
    "\n",
    "  print 'Running tests with p = ', p\n",
    "  print 'Mean of input: ', x.mean()\n",
    "  print 'Mean of train-time output: ', out.mean()\n",
    "  print 'Mean of test-time output: ', out_test.mean()\n",
    "  print 'Fraction of train-time output set to zero: ', (out == 0).mean()\n",
    "  print 'Fraction of test-time output set to zero: ', (out_test == 0).mean()\n",
    "  print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout backward pass\n",
    "In the file `cs231n/layers.py`, implement the backward pass for dropout. After doing so, run the following cell to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx relative error:  1.89290573824e-11\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(10, 10) + 10\n",
    "dout = np.random.randn(*x.shape)\n",
    "\n",
    "dropout_param = {'mode': 'train', 'p': 0.8, 'seed': 123}\n",
    "out, cache = dropout_forward(x, dropout_param)\n",
    "dx = dropout_backward(dout, cache)\n",
    "dx_num = eval_numerical_gradient_array(lambda xx: dropout_forward(xx, dropout_param)[0], x, dout)\n",
    "\n",
    "print 'dx relative error: ', rel_error(dx, dx_num)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fully-connected nets with Dropout\n",
    "In the file `cs231n/classifiers/fc_net.py`, modify your implementation to use dropout. Specificially, if the constructor the the net receives a nonzero value for the `dropout` parameter, then the net should add dropout immediately after every ReLU nonlinearity. After doing so, run the following to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running check with dropout =  0\n",
      "Initial loss:  2.03053296456\n",
      "W1 relative error: 1.31e-08\n",
      "W2 relative error: 4.03e-09\n",
      "W3 relative error: 2.25e-07\n",
      "b1 relative error: 2.92e-10\n",
      "b2 relative error: 9.88e-11\n",
      "b3 relative error: 3.32e-10\n",
      "\n",
      "Running check with dropout =  0.25\n",
      "Initial loss:  2.94959352009\n",
      "W1 relative error: 8.95e-08\n",
      "W2 relative error: 1.98e-08\n",
      "W3 relative error: 2.33e-07\n",
      "b1 relative error: 9.70e-09\n",
      "b2 relative error: 1.50e-10\n",
      "b3 relative error: 3.60e-10\n",
      "\n",
      "Running check with dropout =  0.5\n",
      "Initial loss:  2.35422921859\n",
      "W1 relative error: 9.43e-09\n",
      "W2 relative error: 3.17e-09\n",
      "W3 relative error: 1.18e-06\n",
      "b1 relative error: 1.16e-09\n",
      "b2 relative error: 2.76e-10\n",
      "b3 relative error: 3.02e-10\n",
      "\n"
     ]
    }
   ],
   "source": [
    "N, D, H1, H2, C = 2, 15, 20, 30, 10\n",
    "X = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=(N,))\n",
    "\n",
    "for dropout in [0, 0.25, 0.5]:\n",
    "  print 'Running check with dropout = ', dropout\n",
    "  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,\n",
    "                            weight_scale=5e-2, dtype=np.float64,\n",
    "                            dropout=dropout, seed=123)\n",
    "\n",
    "  loss, grads = model.loss(X, y)\n",
    "  print 'Initial loss: ', loss\n",
    "\n",
    "  for name in sorted(grads):\n",
    "    f = lambda _: model.loss(X, y)[0]\n",
    "    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)\n",
    "    print '%s relative error: %.2e' % (name, rel_error(grad_num, grads[name]))\n",
    "  print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regularization experiment\n",
    "As an experiment, we will train a pair of two-layer networks on 500 training examples: one will use no dropout, and one will use a dropout probability of 0.75. We will then visualize the training and validation accuracies of the two networks over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "(Iteration 1 / 125) loss: inf\n",
      "(Epoch 0 / 25) train acc: 0.156000; val_acc: 0.128000\n",
      "(Epoch 1 / 25) train acc: 0.372000; val_acc: 0.243000\n",
      "(Epoch 2 / 25) train acc: 0.520000; val_acc: 0.274000\n",
      "(Epoch 3 / 25) train acc: 0.568000; val_acc: 0.252000\n",
      "(Epoch 4 / 25) train acc: 0.668000; val_acc: 0.264000\n",
      "(Epoch 5 / 25) train acc: 0.698000; val_acc: 0.311000\n",
      "(Epoch 6 / 25) train acc: 0.758000; val_acc: 0.240000\n",
      "(Epoch 7 / 25) train acc: 0.798000; val_acc: 0.265000\n",
      "(Epoch 8 / 25) train acc: 0.878000; val_acc: 0.272000\n",
      "(Epoch 9 / 25) train acc: 0.904000; val_acc: 0.284000\n",
      "(Epoch 10 / 25) train acc: 0.920000; val_acc: 0.281000\n",
      "(Epoch 11 / 25) train acc: 0.946000; val_acc: 0.297000\n",
      "(Epoch 12 / 25) train acc: 0.924000; val_acc: 0.284000\n",
      "(Epoch 13 / 25) train acc: 0.956000; val_acc: 0.292000\n",
      "(Epoch 14 / 25) train acc: 0.980000; val_acc: 0.304000\n",
      "(Epoch 15 / 25) train acc: 0.978000; val_acc: 0.307000\n",
      "(Epoch 16 / 25) train acc: 0.992000; val_acc: 0.306000\n",
      "(Epoch 17 / 25) train acc: 0.990000; val_acc: 0.303000\n",
      "(Epoch 18 / 25) train acc: 0.994000; val_acc: 0.299000\n",
      "(Epoch 19 / 25) train acc: 0.998000; val_acc: 0.297000\n",
      "(Epoch 20 / 25) train acc: 0.996000; val_acc: 0.293000\n",
      "(Iteration 101 / 125) loss: 0.000000\n",
      "(Epoch 21 / 25) train acc: 0.992000; val_acc: 0.302000\n",
      "(Epoch 22 / 25) train acc: 0.992000; val_acc: 0.307000\n",
      "(Epoch 23 / 25) train acc: 0.988000; val_acc: 0.306000\n",
      "(Epoch 24 / 25) train acc: 0.986000; val_acc: 0.301000\n",
      "(Epoch 25 / 25) train acc: 0.982000; val_acc: 0.306000\n",
      "0.75\n",
      "(Iteration 1 / 125) loss: inf\n",
      "(Epoch 0 / 25) train acc: 0.214000; val_acc: 0.161000\n",
      "(Epoch 1 / 25) train acc: 0.366000; val_acc: 0.242000\n",
      "(Epoch 2 / 25) train acc: 0.432000; val_acc: 0.266000\n",
      "(Epoch 3 / 25) train acc: 0.508000; val_acc: 0.271000\n",
      "(Epoch 4 / 25) train acc: 0.582000; val_acc: 0.315000\n",
      "(Epoch 5 / 25) train acc: 0.648000; val_acc: 0.287000\n",
      "(Epoch 6 / 25) train acc: 0.668000; val_acc: 0.293000\n",
      "(Epoch 7 / 25) train acc: 0.714000; val_acc: 0.306000\n",
      "(Epoch 8 / 25) train acc: 0.746000; val_acc: 0.316000\n",
      "(Epoch 9 / 25) train acc: 0.750000; val_acc: 0.313000\n",
      "(Epoch 10 / 25) train acc: 0.768000; val_acc: 0.300000\n",
      "(Epoch 11 / 25) train acc: 0.792000; val_acc: 0.303000\n",
      "(Epoch 12 / 25) train acc: 0.810000; val_acc: 0.311000\n",
      "(Epoch 13 / 25) train acc: 0.834000; val_acc: 0.335000\n",
      "(Epoch 14 / 25) train acc: 0.824000; val_acc: 0.329000\n",
      "(Epoch 15 / 25) train acc: 0.852000; val_acc: 0.336000\n",
      "(Epoch 16 / 25) train acc: 0.872000; val_acc: 0.322000\n",
      "(Epoch 17 / 25) train acc: 0.882000; val_acc: 0.298000\n",
      "(Epoch 18 / 25) train acc: 0.894000; val_acc: 0.299000\n",
      "(Epoch 19 / 25) train acc: 0.904000; val_acc: 0.318000\n",
      "(Epoch 20 / 25) train acc: 0.906000; val_acc: 0.315000\n",
      "(Iteration 101 / 125) loss: inf\n",
      "(Epoch 21 / 25) train acc: 0.928000; val_acc: 0.323000\n",
      "(Epoch 22 / 25) train acc: 0.926000; val_acc: 0.325000\n",
      "(Epoch 23 / 25) train acc: 0.916000; val_acc: 0.313000\n",
      "(Epoch 24 / 25) train acc: 0.912000; val_acc: 0.309000\n",
      "(Epoch 25 / 25) train acc: 0.938000; val_acc: 0.305000\n"
     ]
    }
   ],
   "source": [
    "# Train two identical nets, one with dropout and one without\n",
    "\n",
    "num_train = 500\n",
    "small_data = {\n",
    "  'X_train': data['X_train'][:num_train],\n",
    "  'y_train': data['y_train'][:num_train],\n",
    "  'X_val': data['X_val'],\n",
    "  'y_val': data['y_val'],\n",
    "}\n",
    "\n",
    "solvers = {}\n",
    "dropout_choices = [0, 0.75]\n",
    "for dropout in dropout_choices:\n",
    "  model = FullyConnectedNet([500], dropout=dropout)\n",
    "  print dropout\n",
    "\n",
    "  solver = Solver(model, small_data,\n",
    "                  num_epochs=25, batch_size=100,\n",
    "                  update_rule='adam',\n",
    "                  optim_config={\n",
    "                    'learning_rate': 5e-4,\n",
    "                  },\n",
    "                  verbose=True, print_every=100)\n",
    "  solver.train()\n",
    "  solvers[dropout] = solver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4MAAAJXCAYAAAApT4F3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3X2Y3HV56P/3PSDqQgJoBRElCGSlJ9XsOMrRn7gstZfS\nUy0EyxGspbBaY1vRaP3Vh/ZgiLZHPL+eROtpC5VNQWo5BYriKa16lJVatSWT2SgBuqlgeFaUp8BW\ngcz9+2MmyeyyG/Yp853deb+ua67M9+kz98x8Mzv3fD7f+xOZiSRJkiSpu5SKDkCSJEmS1H4mg5Ik\nSZLUhUwGJUmSJKkLmQxKkiRJUhcyGZQkSZKkLmQyKEmSJEldyGRQkrToRUQpInZExAuLjkWSpE4R\nzjMoSeo0EbED2PUH6kDgZ8DO5rrVmfk3RcUmSdJiYTIoSepoEXEb8PbMvH4v++yXmTvbGFbbLObn\nJkkqlsNEJUmdLpq3PSsiPhYRV0TE5yPiYeDXI+JVEfHtiHgwIu6OiE9FxH7N/feLiHpEHNVc/lxz\n+3UR8UhE/HNELJv0wRuujIh7I+KBiPh6RBzfsv3ZEbE+IrY3H3s4Ip7R3NbfjOmh5vZfb67/p4g4\nu6WNt0fE9RNi/e2I2Abc0lz/pxFxZ7Otf4mIV7ccv19E/LeI+PeIeDgi/jUinh8RfxERn5jwfP4+\nIn53Du+HJGmRMBmUJC1UpwGXZ+bBwP8GngDeAzwHeA3wBmB1y/4Th8KcBfwBcChwJ/CxvTzWl4Bj\ngecDNwGfa9m2AfgF4JXNx/4IUI+IFwN/D/x/zfVl4Ht7eYyJ8b0JeAXw0ubyd5qP8xzgKuDKXUkn\n8PvA6cDrm6/HO4CfApc2nycAEXEYcBLw+b3EIUnqEiaDkqSF6puZeR1AZv4sM6uZeWM2/AD4SxqJ\nzy4x4firMrPWHIL510DfZA/SbO+yzBzLzMeBdUCl2SNYAn4TOC8zf9Tc91vNNn8duC4zr87MemY+\nkJnfncHz+6PMfDgzf9aM46+by3UaCeZS4Ljmvm8HPpyZtzX3/W5mPpSZ3wZ+GhG7XoezgP+bmQ/O\nIA5J0iJlMihJWqjubF2IiJdExP9pDud8GLgA+Lm9HH9fy/0x4KDJdmpWIv1kRHw/Ih4CttHoxfs5\n4HDgGcBtkxz6IuD70342T3XXhDh+PyJuiYgHgQeAHvY8vxdNEQM0ejHf1rz/Nsb3akqSupjJoCRp\noZo4rPIiGsMwj2kOlfwoT+0NnI2zgVOAgcw8hEZv3K7rGH8IPE5jCOlEd7Kn526ix2gkc7s8f5J9\ndj+/iBgA3gesysxDM/PQZhu7nt8dU8QAjeTv9IjoA46hMeRVkiSTQUnSorEEeDgz/yMifp7x1wvO\ntd2fAQ9GxIHAH9NM1JpDNv8K2BARhzd7Ef+fZuGay4E3RMSqZoGX50bEy5ptjgBvjohnRUQvMDiN\nGJ4AHoiIAyLiAsYnk5cAH4+IYwAiYmVEHNKMcTvwXRrXD17ZHOoqSZLJoCSp4013DqTfA86JiEeA\nPweu2Es7M5lXaSNwL3APjZ7Hb07Y/n4aFT+rwE+AP6IxddMPaBSB+RCNYZ1VGgVgoHHNHzR6Fj/L\nU4duTozvOuBrNIao3gY81Ixpl/8BfAH4WnOI7EXAs1q2X9p87Mum8XwlSV2isHkGI+IS4I3ADzPz\nZVPs82ngl2kMhTknM0faGKIkSYtCc5jpJZk51VBSSVIXKrJncCONst+TiohfBo7NzOU0hvr8RbsC\nkyRpsYiIA4D3AhcXHYskqbMUlgxm5jeBvZW2PpXmcJbM/Bfg4Ig4vB2xSZK0GETEL9AYonoo8KcF\nhyNJ6jD7Fx3AXhzJ+LLhdzfX/bCYcCRJWlgy8yammDJDkqROTganLSKKufBRkiRJkjpEZs5oSqVO\nriZ6N41JdHd5YXPdpDLTm7eOu330ox8tPAZv3ia7eW566+RbEefnpk2b6Om5mkYh1z23np6r2LRp\nU9vi2LlzJ3195wE7W+JorNu5c2dXvRadEMdTY/ior4X/R8hMNm++ib6+8+jpuZqenqvp6zuPzZtv\natvjT3xNNm3aNKuEq+hkcNekvZO5lsZEv0TEq4CHMtMhopIkzbN6vU61WqVarVKv1wuP45577ik0\njiLVajVGRwcY/xWtxOjoSdRqtYKiKka5XKa3dxhoPRfq9PZ+g3K53DUxaLxO+D9Sr9cZHLyIkZEN\njI2dztjY6YyMbGBw8KK2f3bValupVNbQ3799VscXlgxGxOeBbwG9EXFHRJwbEasj4p0AmXkdcHtE\n/DuN+ZJ+p6hYJUlarFq/SPT3b6dSWUOttrXQODZufLjtcfilf49OeS1KpRJDQ6vp61uzu/dl5cr3\nMjS0mlKpPV9hJ8aw//43tz0G6Iz3pBNi6BSdkJDCU5PSWSmiK3O+b42nIXWe66+/vugQpEl5bioz\nc+fOndnXd17CzoRs3hrrdu7cWWAc1xcSx+bNN2Vf33nZ03NV9vRclStXvjs3b76pbY+f2TnvSSe8\nFrvs3LkzN23alJs2bWrrazBZDH/xF39RWAyd8J50Qgyd8H9k06ZN2dNzdcvjN249PVflpk2b2hLD\n5HGQOcM8qrBJ5+dTRORieB6SJLVTtVqlv3/7U35R7um5mhtuOJpKpdJVcUDjl/Zdv+yXy+W29v7s\nUqttZXDwIkZHTwJg+fJhNm58F+XyirbG0QmvhcbrhPekE2Io+v9IvV6nUlnDyMgG9vQO1unrW0O1\nuqFtr8lTPzuDnGEBmUVRTVSSJGk+lEqltiafkymXV1Ctbmj5wv2pQr5wd8JrofE64T3phBiK/j+y\na/jw4OCacQnp0NC7Chg+fCkjI6cx26v/7BmUJKlLdcqv250ShyTNRKf1ko6N/dqMewZNBiVJKlDR\nXyaKHm7VaXFI0kKz6+/IK17xCpNBSZIWij0J0AAAvb3DDA2t7tprwzolDklaiCJmfs2gyaAkSQVw\naKQkaT7NJhn0L40kqe06ZZLzInXKPFWSpO5lMihJaqtOmeRckqRuZzIoSWqber3O4OBFjIxsYGzs\ndMbGTmdkZAODgxd1XQ9hoyT4MND6vOv09n6DcrlcTFCSpAVn12ib2TAZlCS1jUMj99g1T1Vf3xp6\neq6mp+dqVq58L0NDq71eUJI0LbUtNSqrKvSv75/V8U46L0lSQYqeOFmStHDV63UGzx9kpG9k1l18\n/sWRJLWNQyOfqlQqUalUqFQqJoKS9qoTim91QgxqqNVqjC4ZnVNGZ8+gJKltdg2NHBxcM25y8aGh\nd7U9EXJOO0kLSW1LjcHzBxtf/oHeHb0MrRuivLJ9P6R1QgyaX84zKEldpFMSoKLj6JTJ3iVpOur1\nOpVVlfHDAevQN9JH9ZpqWz5DOyEGjfeU92QtzjMoSZpcJ03pUOTQSCuaSlpoJh0OWILRJaNtK77V\nCTG0crhqc7TNuiH6Rvro2dYzuzbmOSZJUgcyAdrDiqaStLC1VtDsX99PZVWF2pbuTEjLK8tUr6ly\nw/tumNXxJoOS1AVMgCRp4SqXy/Tu6J1Ye4veHb1tK77VCTHA+AqaY8vHGFs+xkjfCIPnD7Y1IeuU\nhBT2jLaZ1bHzHIskaQqd8AuirGgqLTSd8NlZdAwThwP2bOthZW0lQ+uG2jbUvhNigM4YrtopCel8\nsJqoJLXBUwuWXNrWgiWNBOhSRkZOo/XK/0YCtKotMXSKTqpoKmnvOqF6ZSfEAHuGAxZZfKsTYugE\nT5eQzraXrgiFVRONiFOADTRexksy88IJ2w8BhoBjgf8ABjPz5inaspqopI5Vr9epVNYwMrLrIw+g\nTl/fGqrVDW37Q7onId2TAG3c+K6uraBZdEVTSXvXCdUrOyEGjdcJ70m1WqV/fT9jy8fGre/Z1sMN\n77uhsGQwIhZGNdGIKAGfAd4ArADOiojjJ+z2EaCWmSuB3wQ+3d4oJWl+dMr1euXyCqrVDdxww9Hc\ncMPRbN78qa5NBMHJ3qWnU/TQyE4YDtgJMWi8Thiu2inXT86HooaJngBsy8ztABFxBXAqcGvLPv8J\n+O8AmflvEXF0RDwvM+9ve7SStEjM5SJzSd2jU4ZGSpMperjqroS09f/I8keWM/Sx9l4/OR8KGSYa\nEW8G3pCZ72wuvw04ITPf07LPHwHPyszfi4gTgG8C/zkzn/IzjMNEJXWyThkmKknT0QnD8Doljk6I\nQZ2r0y43mM0w0U4uIPMJ4FMRsRn4HlADdk6189q1a3ffHxgYYGBgYB+HJ0nTY8ESSTNV5JfMTimO\n0Qm9L50QgzpX0aNthoeHGR4enlMbRfUMvgpYm5mnNJc/BOTEIjITjrkdeGlmPjrJNnsGJXW8TvsF\nUVJnKnqIZqcVx+iEz85OiEF6OrPpGSwqGdwP+DfgdcC9wL8CZ2XmLS37HAyMZeYTEfFbwGsy85wp\n2jMZlDQl/4hLWig6YVhiJ8QgaeYWTDXRzNwJvBv4CrAVuCIzb4mI1RHxzuZuPw/cFBG30Kg6+t4i\nYpW0sNVqW6lU1tDfv53+/u1UKmuo1bYWHZakDmUFzc6o1iipPQqbZ3A+2TMoaTIWbpE0E0UPz4TO\nGqLpqAppYVkww0Tnm8mgpMlUq1X6+7czNnb6uPU9PVdzww1HO8WCpN06ZWhkp8QhaeFZMMNEJUmS\nOkknDM8Eh2hKaq9OnlpCkuakXC7T23spIyOn0foTe2/vNyiXVxUZmiRNqegJtSV1D4eJSlrUarWt\nDA5eNG5+v40b30W5vKLgyCR1EodnSlrovGZQkiZhEQRpYSj6/+rEAjLLH1nOxo9tbGsBGUmaLZNB\nSZK0IHVCJU8oPiGVpNkyGZQkSQuOQzQlae6sJipJkhacTqnkKUndxmRQkiRJkrqQyaAkSQWq1+tU\nq1Wq1Sr1er3ocApRLpfp3dELrU+/3rhusFy2eIsk7Ssmg5IkFaS2pUZlVYX+9f30r++nsqpCbUv3\nDYt0onVJKoYFZCRJKkAnFU3plAqanRKHJC1Esykgs/++CkaSJE3t6YqmVCqV9sTRIVM6QKOHsF3P\nW5LkMFFJkrpWvV5n8PxBRvpGGFs+xtjyMUb6Rhg8f7Brr1+UpG5iMihJUgE6oWiKUzpIUnczGZQk\ndaWiq3haNEWSVDQLyEiSuk4nXSdXZNGUTipiI0mam9kUkDEZlCR1FROg8SYmxssfWc7Gj20sJDGW\nJM3egkoGI+IUYAONP8WXZOaFE7Y/F7gcOALYD/iTzPyrKdoyGZQ6lKXi1Wmq1Sr96/sZWz42bn3P\nth5ueN8NXVnN0v+nkrTwzSYZLOTTPiJKwGeANwArgLMi4vgJu70bGMnMPuBk4E8iwqkwpAWkVttK\npbKG/v7t9Pdvp1JZQ622teiwJE2wa0qHSqViIihJXaSoT/wTgG2ZuT0znwCuAE6dsM99wJLm/SXA\nTzLzyTbGKGkO6vU6g4MXMTKygbGx0xkbO52RkQ0MDl5kyXoVWrylE6p4SpLUCYpKBo8E7mxZvqu5\nrtVfAisi4h5gC/DeNsUmaR7UajVGRweYWLN+dPQkS9Z3udqWGpVVFfrX99O/vp/Kqgq1Le07J6zi\nKUlSQycPu/wwsCUzT46IY4GvRsTLMvPRyXZeu3bt7vsDAwMMDAy0JUhJ0vS1TnK+63eCkXpjkvN2\nFm8pryxTvabqdXKSpAVreHiY4eHhObVRSAGZiHgVsDYzT2kufwjI1iIyEXEd8EeZ+c/N5a8BH8zM\nTZO0ZwEZqcPU63UqlTWMjOyqEwVQp69vDdXqBr94dymLt0iStG8smAIywI3AcRGxLCIOAM4Erp2w\nzy3ALwFExOFAL3BbW6OUNGulUomhodX09a2hp+dqenquZuXK9zI0tNpEUJIkqQMUPbXEp9gztcQn\nImI1jR7CiyPi54CNwFFAAP89M/9mirbsGZQ6lCXr1co5/iRJ2jcW1DyD88lkUJIWDic5lyRp/pkM\nSpL2qlN6ajslDkmSFguTQUnSlCb2yPXu6GVo3ZA9cpIkLQImg5J2s+dFrbxWT5KkxW0hVROVtA/V\nalupVNbQ37+d/v7tVCprqNW2Fh2WClSr1Ro9gq2f+iUYXTK6+0cDSZLUXTp50nlJs1Cv1xkcvGjc\n/H4jI6cxOOj8fpIkSdrDb4XSIlOr1RgdHWBiF9Do6En2ABWsXq9TrVapVqvU6/W2Pna5XKZ3Ry+0\nPmy9cd1guew1g5IkdSOTQUmLXpFJ2C61LTUqqyr0r++nf30/lVUValval5yXSiWG1g3RN9JHz7Ye\nerb1sLK2kqF1Q/YWS5LUpSwgIy0y9XqdSmXNuGGiUKevrzuHiXZCBc1OKt5iYSFJkhYnq4lKAhoF\nZAYHL2J09CQAli8fZuPGd1Euryg4svbqlCSsWq3Sv76fseVj49b3bOvhhvfdQKVSaUsckiRp8ZpN\nMmgBGWkRKpdXUK1uaOkB+lRX9gA9XQVNkzBJktTNTAalRapUKpnsdIhdxVtG6uN7KC3eIkmSitR9\nXQWSukanVNC0eIskSepEXjMoaVGbWEBm+SPL2fixjW0tILOLxVskSdK+YgEZSZqESZgkSVrsTAYl\ndRwTMUmSpH3PaqKSOkonzPEnSZKkydkzKGmf6JQ5/iRJkrrBbHoG/TYmaZ94ujn+JEmSVKzCksGI\nOCUibo2I0Yj44CTbPxARtYjYHBHfi4gnI+KQImKVJEmSpMWmkGGiEVECRoHXAfcANwJnZuatU+z/\nRmBNZv7SFNsdJqqOYtEUh4lKkiS100IaJnoCsC0zt2fmE8AVwKl72f8s4G/aEpk0R7XaViqVNfT3\nb6e/fzuVyhpqta1Fh9V2TrQuSZLU2YrqGXwz8IbMfGdz+W3ACZn5nkn2fTZwF3BsZj40RXv2DKoj\n1Ot1KpU1jIxsoLU7rK9vDdXqhq5MguwllSRJ2vcW69QSbwK+OVUiuMvatWt33x8YGGBgYGDfRiVN\nolarMTo6wMSqKaOjJ1Gr1ahUKgVFVpxSqdSVz1uSJGlfGh4eZnh4eE5tFJUM3g0c1bL8wua6yZzJ\nNIaItiaDkiRJkrSYTewAu+CCC2bcRlHjtW4EjouIZRFxAI2E79qJO0XEwcBJwBfbHJ80K+Vymd7e\nYaDesrZOb+83KJedaF2SJEmdo5CewczcGRHvBr5CIyG9JDNviYjVjc15cXPX04AvZ+Z/FBGnNFOl\nUomhodUMDq5hdPQkAJYvH2Zo6F1eKydJkqSOUkgBmflmARl1GoumSJIkqZ1mU0DGZFCLiknYHr4W\nkiRJ3WMhzTMozTvn99ujtqVGZVWF/vX99K/vp7KqQm1LreiwJEmS1EHsGdSi4Px+e9TrdSqrKoz0\njbS+FPSN9FG9ptpVr4UkSVK3sGdQXevp5vfrJrVajdEloxNfCkaXjHbdayFJkqSpmQxKkiRJUhcy\nGdSi0Gnz+9XrdarVKtVqlXq9/vQHzKNyuUzvjt6JLwW9O3qd61CSJEm7ec2gFo1abSuDgxeNm99v\n48Z3US6vaG8cW2oMnj/YGKpJIwkbWjdEeWX7ErGJMSx/ZDkbP7axrTFIkiSpfZxaQl2v6OkUOql4\nS9GvhSRJktrHZFAqWLVapX99P2PLx8at79nWww3vu4FKpVJQZJIkSVrMrCYqSZIkSZoWk0FpHlm8\nRZIkSQuFyaA0j0qlEkPrhugb6aNnWw8923pYWVvJ0Lohr9mTJElSR/GaQWkfsHiLJEmS2skCMpIk\nSZLUhSwgI0mSJEmaFpNBSZIkSepCc0oGI+K8iDh0voKRJEmSJLXHXHsGDwdujIi/jYhTImJGY1Ql\nSZIkScWYcwGZZgL4euBc4BXA3wKXZOb35x7etGOwgIwkSZKkrlVIAZlmFnZf8/YkcChwVUR8cm/H\nNXsSb42I0Yj44BT7DERELSJuiojr5xqrJEmSJKlhTj2DEfFe4Gzgx8BngS9k5hMRUQK2ZeaxUxxX\nAkaB1wH3ADcCZ2bmrS37HAx8C3h9Zt4dET+XmT+eoj17BiVJkiR1rdn0DO4/x8d8DnB6Zm5vXZmZ\n9Yh4416OO4FGsrgdICKuAE4Fbm3Z563A1Zl5d7PNSRNBSZIkSdLMzXWY6D8AD+xaiIilEfGfATLz\nlr0cdyRwZ8vyXc11rXqB50TE9RFxY0T8xhxjlSRJkiQ1zbVn8M+Bl7csPzrJutnav9nOLwIHAt+O\niG9n5r9PtvPatWt33x8YGGBgYGAeQpAkSZKkzjM8PMzw8PCc2pjrNYMjmdk3Yd13M/NlT3Pcq4C1\nmXlKc/lDNGrRXNiyzweBZ2XmBc3lzwL/kJlXT9Ke1wwKgHq9Tq1WA6BcLlMqzblGkiRJktTxiqgm\neltEvCcintG8vRe4bRrH3QgcFxHLIuIA4Ezg2gn7fBE4MSL2i4ge4D8Dext6qi5X21KjsqpC//p+\n+tf3U1lVobalVnRYkiRJUkeaa8/gYcCnaQzlTOBrwJrM/NE0jj0F+BSNhPSSzPxERKym0UN4cXOf\nD9CYv3An8JeZ+adTtGXPYJer1+tUVlUY6RvZ8xNHHfpG+qheU7WHUJIkSYvabHoG5zzpfCcwGewM\nRQ7RrFar9K/vZ2z52Lj1Pdt6uOF9N1CpVNoWiyRJktRubZ9aIiKeBbwdWAE8a9f6zBycS7taeGq1\nrQwOXsTo6AAAvb2XMjS0mnJ5RbGBSZIkSZrUXLtuPgc8H3gD8A3ghcCOuQalhaVerzM4eBEjIxsY\nGzudsbHTGRnZwODgRdTr9bbEUC6X6d3RC60PV4feHb2Uy+W2xCBJkiQtJHNNBo/LzP8GPJaZlwK/\nQqPQi7pIrVZr9gi2nk4lRkdP2j1sdF8rlUoMrRuib6SPnm099GzrYWVtJUPrhrxeUJIkSZrEXOcZ\nfKL570MR8QvAfcBhc2xTmpXyyjLVa6pOLSFJkiRNw1y/KV8cEYcCf0hjaoibgQv3fogWm3K5TG/v\nMBPHaPb2fqPtQzRLpRKVSoVKpWIiKEmSJO3FrHsGI6IEPJKZDwI3AMfMW1RaUEqlEkNDqxkcXMPo\n6EkALF8+zNDQu0zIJEmSpA4113kGN2XmK+YxntnG4dQSHaDIqSUkSZKkbtb2eQYj4hPAj4H/DTy2\na31mPjDrRmcXh8mgJEmSpK5VRDJ4+ySrMzPbOmTUZFCSJElSN2t7MtgpTAYlSZIkdbPZJINzmloi\nIs6ebH1mXjaXdiVJkiRJ+9Zc5xl8Zcv9ZwGvAzYDJoOSJEmS1MHmdZhoRBwCXJGZp8xbo9N7XIeJ\nSpIkSepabR8mOonHgBfPc5uaBqd1kCRJkjQTc71m8EvAri65EvCfgL+da1CamVptK4ODFzE6OgBA\nb++lDA2tplxeUWxgkiRJkjrWXKeWOKll8Ulge2beNeeoZh5H1w4TrdfrVCprGBnZQCMfB6jT17eG\nanWDPYSSJElSF5jNMNG5Zgp3AP+Smd/IzH8GfhIRR8+xTc1ArVZr9gi2vpUlRkdP2j1stF3q9TrV\napVqtUq9Xm/rY0uSJEmambkmg1cCrd/6dzbXqcvUttSorKrQv76f/vX9VFZVqG1pbzIqSZIkafrm\nmgzun5mP71po3j9gOgdGxCkRcWtEjEbEByfZflJEPBQRm5u3P5xjrItSuVymt3eY8Tl5nd7eb1Au\nl9sSQ71eZ/D8QUb6RhhbPsbY8jFG+kYYPH/QHkJJkiSpQ801Gbw/In5110JEnAr8+OkOiogS8Bng\nDcAK4KyIOH6SXW/IzJc3bx+fY6yLUqlUYmhoNX19a+jpuZqenqtZufK9DA2tbtv1grVajdEloxNH\nqjK6ZLTtQ1UlSZIkTc9cp5Z4F/DXEfGZ5vJdwNnTOO4EYFtmbgeIiCuAU4FbJ+w3owsgu1W5vIJq\ndUPL1BKfsnCMJEmSpL2aUzKYmd8HXhURBzWXH53moUcCd7Ys30UjQZzo1RExAtwN/L+ZefNc4l3M\nSqUSlUqlkMcul8v07uhlpD7SWtCU3h29bRuqKkmSJGlm5jrP4B8Dn8zMh5rLhwK/l5nzcX1fFTgq\nM8ci4peBLwC9U+28du3a3fcHBgYYGBiYhxA0HaVSiaF1QwyeP9gYLgosf2Q5Qx8bsodSkiRJ2geG\nh4cZHh6eUxtznWewlpnlCes2Z+bLn+a4VwFrM/OU5vKHgMzMC/dyzO1AJTMfmGRb184z2Enq9XrL\nUNWyiaAkSZLUJrOZZ3Cu1wzuFxHPzMyfNQN4NvDMaRx3I3BcRCwD7gXOBM5q3SEiDs/MHzbvn0Aj\ncX1KIqjOUeRQVUmSJEkzM9dk8K+Br0XERhrFXs4BLn26gzJzZ0S8G/gKjavMLsnMWyJidWNzXgz8\nWkT8NvAE8B/AW+YYqyRJkiSpaU7DRKExXyDwS0ACjwDPz8zfnYfYZhKDw0QlSZIkda3ZDBOdj4u6\nfkgjETwD+EXglnloU5IkSZK0D81qmGhE9NK4xu9M4EfAlTR6GU+ex9gkSZIkSfvIrIaJRkQd+D/A\n72bmnc11t2XmMfMc33TjcZioJEmSpK7VzmGipwNjwA0R8RcR8Ys0CshIkiRJkhaAuc4zeCBwKo0h\no78IXAZck5lfmZ/wph2HPYOSJEmSutZsegbnXE205cEPpVFE5i2Z+bp5aXT6j20yKEmSJKlrFZoM\nFslkUJIkSVI3K2pqCUmSJEnSAmMyKEmSJEldyGRQkiRJkrrQrCadV+ep1+vUajUAyuUypZJ5viRJ\nkqSpmQwuArUtNQbPH2R0ySgAvTt6GVo3RHllueDIJEmSJHUqq4kucPV6ncqqCiN9I3sG/dahb6SP\n6jVVewglSZKkLmA10S5Uq9UaPYKt72QJRpeM7h42KkmSJEkTmQxKkiRJUhcyGVzgyuUyvTt6od6y\nst64brBc9ppBSZIkSZMzGVzgSqUSQ+uG6Bvpo2dbDz3belhZW8nQuiGvF5QkSZI0JQvILBJOLSFJ\nkiR1r9kUkCksGYyIU4ANNHonL8nMC6fY75XAt4C3ZObfTbFP1yeDkiRJkrrXgqkmGhEl4DPAG4AV\nwFkRcfwU+30C+HJ7I5QkSZKkxa2osYQnANsyc3tmPgFcAZw6yX7nAVcBP2pncJIkSZK02BWVDB4J\n3NmyfFfT1FQQAAAgAElEQVRz3W4R8QLgtMz8c2BG3Z2SJEmSpL3bv+gA9mID8MGW5b0mhGvXrt19\nf2BggIGBgX0SlCRJkiQVbXh4mOHh4Tm1UUgBmYh4FbA2M09pLn8IyNYiMhFx2667wM8BjwHvzMxr\nJ2nPAjKSJEmSutaCqSYaEfsB/wa8DrgX+FfgrMy8ZYr9NwJfspqoJEmSJD3VbJLBQoaJZubOiHg3\n8BX2TC1xS0SsbmzOiyce0vYgJUmSJGkRc9L5OXKyd0mSJElFWzDzDC4WtdpWKpU19Pdvp79/O5XK\nGmq1rUWHJUmSJElPy57BWarX61QqaxgZ+Z/AlubalfT1vZ9qdYM9hJIkSZLaxp7BNqrVatwyehQc\n9UpY1d+4HfVKbhl90e5ho5IkSZLUqTp5nsGOVq/Xefyw/wVn/2BPSv3SER6/7CHq9YECI5MkSZKk\np2fP4FxU7hr/Cpaa6yRJkiSpw5kMzlKpVOKZBzy1Y/WZB+zv9YKSJEmSOp5ZyyyVy2WOf+x4qLes\nrMPxjx1PuVwuLC5JkiRJmg6TwVkqlUoMrRuib6SPnm099GzrYWVtJUPrhuwZlCRJktTxnFpijpx0\nXpIkSVLRZjO1hMmgJEmSJC1wzjMoSZIkSZoWk0FJkiRJ6kImg5IkSZLUhUwGJUmSJKkLmQxKkiRJ\nUhcyGZQkSZKkLmQyKEmSJEldyGRQkiRJkrqQyaAkSZIkdaHCksGIOCUibo2I0Yj44CTbfzUitkRE\nLSI2RcQvFhGnJEmSJC1GkZntf9CIEjAKvA64B7gRODMzb23Zpyczx5r3Xwpck5nHTdFeFvE8JEmS\nJKkTRASZGTM5pqiewROAbZm5PTOfAK4ATm3dYVci2HQQ8OM2xidJkiRJi1pRyeCRwJ0ty3c1140T\nEadFxC3AdcB72hSbJEmSJC16+xcdwN5k5heAL0TEicDngJdMte/atWt33x8YGGBgYGBfhydJkiRJ\nhRgeHmZ4eHhObRR1zeCrgLWZeUpz+UNAZuaFeznm+8AJmfmTSbZ5zaAkSZKkrrWQrhm8ETguIpZF\nxAHAmcC1rTtExLEt918OMFkiKEmSJEmauUKGiWbmzoh4N/AVGgnpJZl5S0SsbmzOi4E3R8TZwOPA\nY8BbiohVkiRJkhajQoaJzjeHiUqSJEnqZgtpmKgkSZIkqUAmg5IkSZLUhUwGJUmSJKkLmQxKkiRJ\nUhcyGZQkSZKkLmQyKEmSJEldyGRQkiRJkrqQyaAkSZIkdSGTQUmSJEnqQiaDkiRJktSFTAYlSZIk\nqQuZDEqSJElSFzIZlCRJkqQuZDIoSZIkSV1o0SSD9Xq96BAkSZIkacFYNMlgZVWF2pZa0WFIkiRJ\n0oKwaJLBkb4RBs8ftIdQkiRJkqZh0SSDlGB0ySi1mr2DkiRJkvR0CksGI+KUiLg1IkYj4oOTbH9r\nRGxp3r4ZES8tIk5JkiRJWowKSQYjogR8BngDsAI4KyKOn7DbbUB/Zq4EPg785V4brUPvjl7K5fI+\niFiSJEmSFpeiegZPALZl5vbMfAK4Aji1dYfM/E5mPtxc/A5w5N4aXFlbydC6IUqlxTPyVZIkSZL2\nlf0LetwjgTtblu+ikSBO5R3AP+ytwc1f2GwiKEmSusLRRx/N9u3biw5DUgGWLVvGD37wg3lpq6hk\ncNoi4mTgXODEve23bt263fcHBgYYGBjYt4FJkiQVZPv27WRm0WFIKkBEADA8PMzw8PDc2irigyQi\nXgWszcxTmssfAjIzL5yw38uAq4FTMvP7e2kv/UCUJEndIiJMBqUuNdX//+b6mElbRY2rvBE4LiKW\nRcQBwJnAta07RMRRNBLB39hbIihJkiRJmrlCholm5s6IeDfwFRoJ6SWZeUtErG5szouB/wY8B/iz\naPSFPpGZe7uuUJIkSZI0TYUME51vDhOVJEndxGGiUvdaDMNEJUmSNM/q9TrVapVqtUq9Xi+sjU6w\nfft2SqXSgn4O88lzYw/PjT1MBiVJkhaBWm0rlcoa+vu309+/nUplDbXa1ra38eCDD7Jq1SoOOugg\nXvziF/M3f/M3e91//fr1HHHEERxyyCG84x3v4Iknnph1WxPtqrrYaUqlErfddlvbHq+2pUZlVYX+\n9f30r++nsqpCbUut7W3M5P387d/+bZYsWcLSpUtZunQpz3rWszj44IN3bx8YGODZz342S5cuZcmS\nJfz8z//8jGLx3GhwmKgkSdICM3GYWL1ep1JZw8jIBvb81l+nr28N1eqGac3FPB9tAJx11lkADA0N\nsXnzZn7lV36Fb3/725N+Wf/yl7/MOeecw/XXX88RRxzBaaedxqtf/Wr++I//eMZtTbR9+3aOOeYY\nnnjiiUljz8zCEoL99tuPbdu2ccwxx+zzx6rX61RWVRjpG2l9W+kb6aN6TXX658Yc24C5vZ/nnnsu\n++23H5/97GcBOPnkkzn77LM599xzp/XYrRb6ueEw0UnYzStJkrpVrVZjdHSA8V/tSoyOnkStNr3e\nm/loY2xsjL/7u7/j4x//OM9+9rN5zWtew6mnnsrnPve5Sfe/7LLLePvb387xxx/PwQcfzPnnn8/G\njRtn1Va9XucDH/gAz3ve8zjuuOP4+7//+3HbTz75ZP7wD/+QE088kQMPPJDbb7+de++9l1NPPZXn\nPve59Pb27k40AC644ALOOOMMzjzzTJYuXcorXvEKvvvd7+7efuutt3LyySdz6KGH8tKXvpQvfelL\n4x5raGho9/Kll17Ka1/7WgBOOukkMpOXvexlLF26lCuvvHJar+1s1Wo1RpeMTnxbGV0yOrNzY45t\nzPT9bPXYY49x9dVXc84554xbP93OIM+NqS2aZHA2wxgkSZI0f0ZHR3nGM57Bscceu3vdypUr2bp1\n8u9oW7duZeXKleP2/dGPfsSDDz4447YuvvhirrvuOrZs2cKmTZu46qqrnrLP5Zdfzmc/+1l27NjB\nUUcdxZlnnslRRx3Ffffdx5VXXslHPvKRcZN4X3vttbzlLW/hwQcf5KyzzuK0005j586dPPnkk7zp\nTW/ilFNO4f777+fTn/40v/7rv862bdumfG129TR94xvfAOB73/sejzzyCGecccaUxywmM30/W119\n9dUcdthhnHjiiePWf/jDH+awww7jta997e7XdTKeG1NbNMngyMgGBgcvsodQkiR1nXK5TG/vMND6\nPahOb+83KJfLbWvj0UcfZenSpePWLV26lB07dky5f+t1YEuXLiUz2bFjx4zbuvLKK1mzZg0veMEL\nOOSQQ/jwhz/8lH3OOeccjj/+eEqlEvfddx/f+ta3uPDCC3nGM57BypUrecc73sFll122e/9KpcKq\nVavYb7/9eP/738/PfvYzvvOd7/Cd73yHxx57jA9+8IPsv//+nHzyybzxjW+c0TWN7brEqVwu07uj\nd+LbSu+O3pmdG3NsY6bvZ6vLLruMs88+e9y6T37yk9x2223cfffd/NZv/RZvetObuP322yc93nNj\naosmGZzpMAZJkqTFolQqMTS0mr6+NfT0XE1Pz9WsXPlehoZWT/t6rvlo46CDDuKRRx4Zt+7hhx9m\nyZIl09r/4YcfJiJYsmTJjNu65557eNGLXrR7edmyZU/Zp3X7Pffcw3Oe8xx6enrGHXP33XdPun9E\ncOSRR3LPPfc85bEmO7ZTlEolhtYN0TfSR8+2Hnq29bCytpKhdUMzOzfm2MZM389d7rjjDoaHh5+S\nDL7yla/kwAMP5BnPeAZnn302r3nNa7juuusmbcNzY2qFTDovSZKk+VUur6Ba3bD7h/Fy+VPT/qI+\nX2309vby5JNP8v3vf3/3cMAtW7awYsWKSfdfsWIFW7Zs4dd+7dcAGBkZ4fDDD+fQQw/lmc985oza\nOuKII7jzzjt3L2/fvv0p+7QWBXnBC17AAw88wGOPPcaBBx4INBKPI488cvc+re1lJnfddRcveMEL\nyEzuuOOOcW3fcccdvOQlLwHgwAMPZGxsbPe2++67b9KY26W8skz1mmrL+1qe+bkxxzZmem7scvnl\nl3PiiSdy9NFH73W/vc296bmxF5m54G9Aws7s6zsvd+7cmZIkSYtZ4ytcZzrrrLPyrW99az722GP5\nT//0T3nIIYfkzTffPOm+//iP/5hHHHFE3nzzzfnAAw/kwMBAfuQjH5lVW3/+53+eK1asyLvuuisf\neOCBfN3rXpelUmn3d8OBgYG85JJLxh3T39+f5513Xv70pz/NLVu25OGHH55f//rXMzNz7dq1ecAB\nB+Q111yTTz75ZP7Jn/xJvvjFL84nn3wyH3/88Tz22GPzwgsvzCeeeCKvv/76XLp0aY6OjmZm5h/8\nwR/kySefnGNjY7lt27Zcvnx5vva1r939uEcccUR+9atfnf2LvEDN5P3c5SUveUn+1V/91bh1Dz30\nUH75y1/On/70p/nkk0/m5ZdfngcddFBu27Zt0jYW27kx1f//5vqZ5VEzPaATb0CuXPnu3Lz5pr2+\ncJIkSYtBJyeDDzzwQJ522ml54IEH5rJly/KKK67Yve2OO+7IJUuW5J133rl73fr16/Pwww/Pgw8+\nON/+9rfn448/Pq22JnryySfz/e9/fz73uc/NY445Jv/sz/5s3Bf+k08++Slf+O++++584xvfmM95\nznPyuOOOy4svvnj3trVr1+YZZ5yRZ555Zi5ZsiRf/vKX58jIyO7tN998c5500kl58MEH54oVK/KL\nX/zi7m0//vGP8/Wvf30uXbo0TzzxxLzgggvGfeG/6KKL8ogjjshDDz00r7zyypm8vAvaTM+Nb3/7\n23nQQQflo48+Oq6d+++/P1/5ylfm0qVL89BDD81Xv/rV+bWvfW3Kx11s58Z8JoOLZp7BnTt3zri7\nW5IkaSHa25A4zY8LLriA73//++OKhkhQ/LnhPIOTMBGUJEmSpOkzg5IkSZKkLrRohokuhuchSZI0\nHQ4TlbqXw0QlSZIkSXNiMihJkiRJXchJ5yVJkhaYZcuWjZskW1L3WLZs2by15TWDkiRJkrTALahr\nBiPilIi4NSJGI+KDk2x/SUR8KyJ+GhHvLyJGaa6Gh4eLDkGalOemOpnnpzqV56YWm0KSwYgoAZ8B\n3gCsAM6KiOMn7PYT4Dzgf7Q5PGne+EdDncpzU53M81OdynNTi01RPYMnANsyc3tmPgFcAZzaukNm\n/jgzq8CTRQQoSZIkSYtZUcngkcCdLct3NddJkiRJktqgkAIyEfFm4A2Z+c7m8tuAEzLzPZPs+1Fg\nR2b+z720Z/UYSZIkSV1tpgVkippa4m7gqJblFzbXzcpMn7QkSZIkdbuihoneCBwXEcsi4gDgTODa\nvexvsidJkiRJ86iweQYj4hTgUzQS0ksy8xMRsRrIzLw4Ig4HNgFLgDrwKPCfMvPRQgKWJEmSpEVk\nUUw6L0mSJEmamcImnZ8PTzdxvVSkiPhBRGyJiFpE/GvR8ah7RcQlEfHDiPhuy7pDI+IrEfFvEfHl\niDi4yBjVvaY4Pz8aEXdFxObm7ZQiY1R3iogXRsTXI2JrRHwvIt7TXO/npwo1ybl5XnP9jD87F2zP\nYHPi+lHgdcA9NK5DPDMzby00MKkpIm4DKpn5YNGxqLtFxIk0htpflpkva667EPhJZn6y+WPaoZn5\noSLjVHea4vx82kri0r4WEc8Hnp+ZIxFxEFClMS/2ufj5qQLt5dx8CzP87FzIPYNPO3G9VLBgYf8f\n0yKRmd8EJv4ocSpwafP+pcBpbQ1Kapri/ASLx6lgmXlfZo407z8K3EKjAr6fnyrUFOfmrjnbZ/TZ\nuZC/qDpxvTpdAl+NiBsj4reKDkaa4LDM/CE0/qgAhxUcjzTRuyNiJCI+6zA8FS0ijgb6gO8Ah/v5\nqU7Rcm7+S3PVjD47F3IyKHW612Tmy4H/AvxucyiU1KkW5jUDWqz+DDgmM/uA+wCHi6owzWF4VwHv\nbfbCTPy89PNThZjk3JzxZ+dCTgbndeJ6ab5l5r3Nf+8HrqExtFnqFD9sTuGz69qDHxUcj7RbZt6f\ne4oa/CXwyiLjUfeKiP1pfNn+XGZ+sbnaz08VbrJzczafnQs5GZzpxPVS20RET/PXGiLiQOD1wE3F\nRqUuF4y/juBa4Jzm/d8EvjjxAKmNxp2fzS/Yu5yOn58qzhBwc2Z+qmWdn5/qBE85N2fz2blgq4nC\n5BPXFxySBEBEvJhGb2AC+wN/7fmpokTE54EB4LnAD4GPAl8ArgReBGwH/mtmPlRUjOpeU5yfJ9O4\nBqYO/ABYvesaLaldIuI1wA3A92j8PU/gI8C/An+Ln58qyF7Ozbcyw8/OBZ0MSpIkSZJmZyEPE5Uk\nSZIkzZLJoCRJkiR1IZNBSZIkSepCJoOSJEmS1IVMBiVJkiSpC5kMSpIkSVIXMhmUJKkpInZGxOaI\nqDX//f15bHtZRHxvvtqTJGmu9i86AEmSOshjmfnyfdi+k/tKkjqGPYOSJO0Rk66MuD0iLoyI70bE\ndyLimOb6ZRHxtYgYiYivRsQLm+sPi4i/a66vRcSrmk3tHxEXR8RNEfGPEfHMNj0vSZKewmRQkqQ9\nnj1hmOgZLdsezMyXAf8L+FRz3Z8CGzOzD/h8cxng08Bwc/3Lga3N9cuBP83MXwAeBt68j5+PJElT\nikxHrEiSBBARj2Tm0knW3w6cnJk/iIj9gXsz83kRcT/w/Mzc2Vx/T2YeFhE/Ao7MzCda2lgGfCUz\nX9Jc/n1g/8z847Y8OUmSJrBnUJKk6ckp7s/Ez1ru78Rr9yVJBTIZlCRpj0mvGWx6S/PfM4FvN+//\nM3BW8/7bgH9q3v+/wO8AREQpInb1Nu6tfUmS2spfJCVJ2uNZEbGZRtKWwD9m5kea2w6NiC3AT9mT\nAL4H2BgRHwDuB85trl8DXBwRbweeBH4buA+riUqSOojXDEqS9DSa1wxWMvOBomORJGm+OExUkqSn\n5y+nkqRFx55BSZIkSepC9gxKkiRJUhcyGZQkSZKkLmQyKEmSJEldyGRQkiRJkrqQyaAkSZIkdSGT\nQUlSV4iIZRFRjwj/9kmShMmgJGmBiIh/iIi1k6w/NSLunWaS53xKkiQ1mQxKkhaKS4G3TbL+bcDn\nMrPe5njmVURE0TFIkrqLyaAkaaH4AvDciDhx14qIOAR4I3BZc/m/RMTmiHg4IrZHxEen23hEfDAi\n/j0iHomImyLitAnbfysibm7Z3tdc/8KIuDoifhQR90fEp5vrPxoRn2s5ftww1Yi4PiI+HhHfjIjH\ngBdHxDktj/HvEfHOCTGcGhG15vPbFhGvj4hfi4hNE/Z7f0RcM93nLknqTiaDkqQFITN/ClwJnN2y\n+i3ALZl5U3P5UeA3MvNg4FeAd0XEr07zIf4deE1mLgUuAC6PiMMBIuIM4Hzgbc3tvwr8pJnY/R/g\nduAo4EjgitawJz6NCctvA94BLAHuAH4I/JfmY5wLrG9JOk+g0Tv6e83n1w/8ALgWODoiXjKh3Uun\n+bwlSV3KZFCStJBcCpwREQc0l3+DlqQnM2/IzK3N+zfRSMxOmk7DmXl1Zv6wef9KYBtwQnPz24FP\nZubm5vbbMvPO5vYjgN/PzJ9m5uOZ+a0ZPJ+/ysxbM7OemU9m5j9k5g+aj/FPwFeA1zb3HQQuycyv\nN7ffm5mjmfk48Lc0h9BGxApgGfD3M4hDktSFTAYlSQtGZv4zcD9wWkQcA7wS+Pyu7RFxQkR8vTlk\n8yFgNfBz02k7Is5uDsF8MCIeBFa0HPsi4PuTHPYiYPscrle8c0IMvxwR346InzRj+OVpxACNYbJv\nbd5/G/C3mfnELGOSJHUJk0FJ0kLzOeA3aSQ9X87M+1u2fZ7GtYVHZuYhwEXA0xZmiYijgIuB38nM\nQzPzUGBry7F3AsdOcuidwFFTVDJ9DOhpWT5ikn12Dxtt9nZeBXwSeF4zhn+YRgxk5neAxyPitTSS\nws9Ntp8kSa1MBiVJC81lwC/RuNZu4nVxBwEPZuYTzWvs3jph+1SJ4YFAHfhxRJQi4lzgF1q2fxb4\nQES8HCAijo2IFwH/CtwLfCIieiLimRHx/zSPGQH6I+JFEXEw8KGneV4HNG8/zsx6RPwy8PqW7ZcA\n50bEydHwggnXCV4OfAaY6VBVSVKXMhmUJC0ombkd+BaNXrdrJ2z+HeBjEfEw8IfA/554+BRt3gL8\nCfAd4D4aQ0S/2bL9KuCPgM9HxCPANcBzmsND3wQsp1EA5k7gvzaP+b/Nx/8ucCPwpb3FkpmPAu8B\nroyIB4AzgS+2bL+RRlGZDcDDwDCNojW7fI5GAmuvoCRpWiJz386/GxGn0PjDVaJx4fuFE7b/KvAx\nGr/I7qRxEf7Xm9t+QOMPXh14IjNPQJIkPUVEPItGNdKXZ+ZU1xZKkrTbPk0Gm9dQjAKvA+6h8cvo\nmZl5a8s+PZk51rz/UuCazDyuuXwbUMnMB/dZkJIkLQIR8X4a01L8UtGxSJIWhv33cfsnANuaQ3qI\niCuAU4HdyeCuRLDpIODHLcuBQ1klSdqriLi9efe0QgORJC0o+zoZPJLxZbPvYs+cTbtFxGnAfwee\nD7yhZVMCX42IncDFmfmX+zBWSZIWpMx8cdExSJIWnn2dDE5LZn4B+EJEnEjjwvdd1dFek5n3RsTz\naCSFt2TmNyceHxH79sJHSZIkSepwmfm00ym12tfJ4N2Mr3T2wua6SWXmNyNi/4h4bmb+JDPvba6/\nPyKuodGr+JRksLnPPIYtzY+1a9eydu3aosOQnsJzU53M81OdynNTnSxiRnkgsO+vx7sROC4iljUn\n0z2TCWXAI+LYlvsvB8jMnzTnazqouf5AGnMt3bSP45UkSZKkrrBPewYzc2dEvBv4CnumlrglIlY3\nNufFwJsj4mzgceAx4C3Nww8HrmkOAd0f+OvM/Mq+jFeSJEmSusU+v2YwM/+RPdcA7lp3Ucv9TwKf\nnOS424G+fR2ftC8NDAwUHYI0Kc9NdTLPT3Uqz00tNvt80vl2iIhcDM9DkiRJkmYjImZcQMY5/CRJ\nkiSpC5kMSpIkSVIXMhmUJEmSpC7UEZPOS5Ik1et1arUaAOVymVLJ36wlaV8yGZQkqUAmQA21LTUG\nzx9kdMkoAL07ehlaN0R5ZbngyCRp8bKaqCRJBTEBaqjX61RWVRjpG9lzAUsd+kb6qF5T7doEWZJm\nwmqikiQtEPV6ncHzBxnpG2Fs+Rhjy8cY6Rth8PxB6vV60eG1Va1WayTErd9KSjC6ZHR3r6kkaf6Z\nDEqSVAATIElS0UwGJUlSocrlMr07eqG1Q7TeGDZbLnfXkFlJaieTQUmSCmACtEepVGJo3RB9I330\nbOuhZ1sPK2srGVo35PWCkrQPWUBGkqSCTCwgs/yR5Wz82Ma2F5DplIqmnRKHJC1EsykgYzIoSVKB\nik6ArGgqSYuDyaAkSZo2p3SQpMXDqSUkSdK0WdFUkrrb/kUHIEmSJGlhKHpou+aX754kSV3KiqaS\nZqK2pUZlVYX+9f30r++nsqpCbYujCBay/7+9+4+Pq67zPf7+jIUtaRNaXCltlQIlqVjWZJiFyy4l\nTZf70N59ILS4XFpULgTcgisS8T5EYBcK7u5jYR+7gKuwVEmFRekKtcq9D1bgiqG42hWGmailNLFI\nSqnVYitNG5Am87l/zCSZpEkzk2bmnMx5PR+PPDrnZz6dnDlzPuf7PZ8vzwwCABBhYaloCmBsQbbK\n8Yxx+FFABgAAFI1uX4N4LxBWQVf+TSaTaryrUT21PUPmV3VWaeNnNyqRSJQlDoxuPMkgzwwCABBx\nsViMCzkFf7ENjCaTyaj5luYhrXLpTFrNtzTTKocjwpEDAAAiL/9iu6e2Rz21PUo3ZC+2M5nM2DsA\nSigMlX95xrgykQwCAIDIC8PFNhBmsVhMrbe3qiHdoKrOKlV1Vqk+Va/W21tpmZzE6CYKAACA0OI5\nzsFWuXRmaPGWcrfKxevjSm5IRv7vUUkoIAMAACKPSonhxHOcg6j8i7FQTRQARsBdZQCF4GI7XEjQ\nD8X3GQ6HZBCRx0kSw3FXGUAx+B4JjzANZcBxgcmAoSUQaVz0YzhKcQMoFsNsYDiuL1DJaBlERaAr\nCUYSprvKEneWAaAYYfhuD0MMGBnfqYcaT8sg7xoqAiXBEXap9pQSyxNqvKtRjXc1KrE8oVQ7xyYA\njCYMQxlwfRFOfKdOHLqJAiXA3apwCEspbrqrhhOfUyD8GMoAw/GdOrF4t1AR+i/6lcmbGcBFvySl\nUpuVSLSosbFLjY1dSiRalEptLmsMyArDXWWJO8thxF1lYPLof44zkUiU/UI/TNcXyOI7dWLRMoiK\n0H/RP7wkeOsXy3vRn8lk1Nx8v9Lpu9V/lkqnl6m5uUXJ5N3crQoAd5UxHHeVARQqLNcXCKdK6GFC\nARlUlKA/lMlkUo2NXerpuWjI/Kqq9dq48SQq1EUUBQjCJWyFhQCEX9DXFxgUlu/UMFaZZWgJRB4l\nwRFG3FkGCsMFN8KK64tBQX9Ow/CdWkk9TGgZBCZQJpNRItEypJuolFFDQ3S7iQb9pREmvBfhEJa7\nyhgqldqs5ub71dHRJEmqq2tTa+sqxeMLyxoHn1NgdGH5nEpSb2+v1q1bJ0lasWKFpkwpXxtX2HqY\n9J+3/viP/5ihJYAgxWIxtbauUkNDi6qq1quqar3q669Ta+uqSF5QUKRjqCCLIGBQWAoLYVD+89Y9\nPRepp+cipdN3q7n5fmUymbF3MEEoAAaMLiyfUyn7WT3zzOu1alWVVq2q0plnXh/Zz2oqtVlnnHGd\nFi16Zlzb0zIIlAB3lml9QfjxOR0U9HsRhuet6dkBHF4YPqdSOD6rYbnGyWQyOu30j6njwBYp0Slt\n6KFlEAgDWoAo/YzDy2QySiaTSiaTZb+j3I/PaRatYVmpVCrX9W3oSaujY3Eg56wwfEaAMArDZzUs\nPUySyaQ639okXd4u1feMvcEIovvtBwAIBN2HwyMs3b7i8bjq6to0fDC3urpnIzmWGwk6RhPkTQI+\np+SV++sAACAASURBVEP1D1218bMbtfGzG/Xid14seyXRrVu3yuM7jyijIxkEUBIM1IuR5Fdg66nt\nUU9tj9IN2QpstH6UXxjusEvheN46LBe6YUnQET5B3yQIw+dUCs9nNQwWLFggix3ZeaHkfzkzW2pm\nL5tZh5ndMMLyC8ys3cxSZvaCmf1ZodsCCK+wdKHAUEF3PaP7MEYTjy9UMnm3Nm48SRs3nqQXX7yn\nrBUKw3KhG5YEHeESlpsEQX9OpTB9VoNvwU8kEqrdN39oXlykkhaQMbOYpA5J50naKel5SSvc/eW8\ndarcvSf3+o8kbXD3UwvZNm8fFJABQirowhQYFIYBcsNWjjvqwlCIIWyCPmeFpUiHFGzpfgwVpuMi\nLIL8rIbp3JlqT+mKW67Q1mlb9fYjb4eugMxZkjrdvcvdD0paJ+nC/BX6E8Gc6ZLeKHRbAOFHkY5w\nCEv3TLoPh0tY7rCHSdDnrMEucL2Skrmf3rJ3gXvk3x9Tzemz9YlvX6lPfPtK1Zw+W4/8+2Nl+/3A\nWIL8rIapBT9eH9eLG17UDz/3w3FtX+p3bq6k1/Kmd+TmDWFmy8xsi6QnJH2mmG0BAGMLS/fM/u7D\n9el6Td06VVO3TqX7cMDC0O0Lg2KxmD7/hUYds2C2tPxPpeV/qqkLZuvzX2gs22ekt7dXV956jd66\n5A2p/h2p/h29dckbuvLWa9Tb21uWGPIF3b09DHhODofTnxiPRyja+939O5K+Y2bnSvo3SQuK3cfq\n1asHXjc1NampqWmiwgMATKTM0VLXudLWj2WnF2zPzkNgjuRCAhMrk8nozm/+XTYRy+V+b//RG7rz\nm3+nSy6+qCwJ4bp16/TWB/YdcvPordP2ad26dfr4xz9e8hj6pVKb1dx8f64VRqqre1Ctrasid8Oi\nvxW/ublFHR2LJUm1tW1qbb2aG2kByCbnDyqdXqb8bqLZ5Hx52eJoa2tTW1vbEe2j1Mng65JOzJt+\nb27eiNz9OTObYmbvLnbb/GQQADBUf/fMdGboALnl7p7ZXwShvf0e9QfS3p5Rc3N0n1ED8o3Vih+l\npD2/aEr/G5JOL4vs+aK/FX/wObl7IvcehEVYkvPhDWC33XZb0fsodbTPSzrVzOaZ2dGSVkh6PH8F\nM5uf9/oMSXL33xayLQCgMGGp7hqm5ywAjGzFihU65qWaQ57tPWZLjVasWFG2ODhfHCroZ1oxqFK6\n2Je0ZdDd+8zs05KeUvaT/IC7bzGzVdnFvkbSR83sMknvSDqgbNI36raljBeoJEFXxEP49A+Qy3EB\nhFcYWvGnTJmiB267L/vc4Gn7JElTt1Trgdvui2xFUb5TMZJK6GJf0qElyoWhJYChwjCEADCSMJXj\nDgsuMjHc8HN47b5arf3i2rKfw4MeWiIs54tDn1tsi+Rziwg/Myt6aAmSQaDCZDIZJZYnlG4Yele5\nId2g5IYkF5oI3OCF1eBzFmvXXh3JCysuMjEabhJkBX2+CEtCChSCZBAAg3pjUuBCl4tMoFBBni8Y\n7B2TyXiSwWh2/AYABKoSnrM4UmMVx4j6+wP043wBlA63HYEK0198YHgVuHIPIQAAwGTHYO+odCSD\nQAlkMhklk0klk0llMpmxN5hAYRlCAMDhcZEJhF//eHINDS2qqlqvqqr1qq+/Tq2tq/hORUXgmUFg\ngoWlkifPZAHhF3RxDACF4TsVkwEFZICAUckTQLG4yAQATAQKyAABS6VS2RbBofUg1FHdQUEIcNEf\nMmH5e1AcAwAQFK5EAKAMUqnNSiRa1NjYpcbGLiUSLUqlNgcdVmTx9wAAgG6iwISimyhGwnhy4cLf\nAwBQicbTTZRvPGACUckTIxlrPLlyCrLSbViE6e8BAECQeGYQmGDx+riSG5KheBYJyDdYubJJklRX\n96BaW1dRuRIAgIiimygAlFgYuiWGIYaw4L0AAFQiuokCQAiFYdBiukYOCsPfAwCAMKCbKACUQTy+\nUMnk3Xndh+8h8QgQfw8AAOgmCgCRQNdIAAAq23i6iZIMVoiwDJ4MILwGC8gsliTV1rZp7dqrKSAD\nAEAFIBmMqEMrBLZRIRDAiLhxBABAZSIZjCC6fgFjIwECAACVjmqiERSmCoEMZo0wSqU2K5FoUWNj\nlxobu5RItCiV2hx0WAAAAIGjmigmRKo9peZbmtVR3SFJquuuU+vtrYrXxwOODFGWyWTU3Hz/kJbz\ndHqZmptpOQcAAOBKaJKLx+Oqq2uTlN8Sl1Fd3bOKx8uTiGUyGTXf0qx0Q1o9tT3qqe1RuiGt5lua\naSFEoMLUcg4AABA2JIOTXBgGT06lUtkWwaHX2+qo7uCCGwAAAAgpuolWAAZPBkaWbTl/UOn0MuUX\nWMq2nC8PMjQAAIDAkTHgiMXjcdV11w3vqaq67rqydVUFRhKGlnMAAICwYmiJChCG4i3DY6jdV6u1\nX1xLARmEYliHMMQAAABQSowzGEGZTEaJ5QmlG9L5veDUkG5QckOyrBe9YbjgDkMMGJRKbVZz8/25\nIi5SXV2bWltXKR5fGGhcAAAAlYZkMIKSyaQa72pUT23PkPlVnVXa+NmNSiQSAUVWfiQe4ZLJZJRI\ntAwZ1kHKqKGBYR0AAAAmGoPOI7Lyx5Pr6blIPT0XKZ2+W83N9zO8RUAY1gEAACDcSAYnOYq3ZJF4\nAAAAAMUhGZzkYrGYWm9vVUO6QVWdVarqrFJ9ql6tt7fSDQ+Byg7r0KbhdyqywzpE50YFAABAWPHM\nYIWIeuEUnk8Lp8HnOBdLkmpr27R27dU8xwkAADDBKCCDSCPxCKeo36gAAAAoB5JBRB6JBwAAAKKI\nZBAAAAAAIoihJQAAAAAABSEZBAAAAIAIIhkEAAAAgAiaEnQAqBwUbwEAAAAmD67WMSFSqc1KJFrU\n2NilxsYuJRItSqU2Bx0WAAAAgFGUvJqomS2V1D8S+APufsew5ZdKuiE32S3pU+7+09yyVyW9KSkj\n6aC7nzXK76CaaIAY8B0AAAAIVuiqiZpZTNKXJX1Y0kJJK83s/cNWe0VSo7vXS/pbSWvylmUkNbl7\nfLREEMFLpVLq6GjS0MMppo6OxQPdRgEAAACES6mbbM6S1OnuXe5+UNI6SRfmr+Dum9z9zdzkJklz\n8xZbGWIEAAAAgMgpdaI1V9JredM7NDTZG+4qSf+RN+2Snjaz583skyWIDxMgHo+rrq5N2YbcfhnV\n1T2reDweTFAAAAAADis01UTNbImkKyQtypt9jrv/yszeo2xSuMXdfzjS9qtXrx543dTUpKamphJG\ni3yxWEytravU3Nyijo7FkqTa2ja1tl7N84IAAABACbS1tamtre2I9lHSAjJmdrak1e6+NDf9BUk+\nQhGZD0paL2mpu28bZV+3Sup2938eYRkFZEKAoSUAAACAYIyngEypk8F3Sdoq6TxJv5L0E0kr3X1L\n3jonSvq+pE+4+6a8+VWSYu6+38ymSXpK0m3u/tQIv4dkEAAAAEBkjScZLGk3UXfvM7NPK5vI9Q8t\nscXMVmUX+xpJfyPpOEn3mplpcAiJWZI2mJnn4vzGSIlg0GgNAwAAADAZlXycwXIIqmUwldqsK674\nV23deqIkacGC7Vq79mrF4wvLHgsAAACA6CpJN1Ezu1bSw+6+90iCK6UgksFMJqPTTv+YOg5skRKd\n2ZnJWtVNO01bfv4NWggBAAAAlE2pBp2fJel5M/uWmS3NdeWMvGQyqc63NkmXt0v1Pdmfy9vV+dYm\nJZPJoMMDlMlklEwmlUwmlclkxt4AAAAAkTJmMujufy2pVtIDki6X1Glmf29m80scW6ht3bpVHt85\n9B2MSd6wU1u3bg0sLkDKdmFOJFrU2NilxsYuJRItSqU2Bx0WAAAAQqSgvoy5Ppi7cj+9kmZKeszM\n7ixhbKG2YMECWezQ1hZ7V0YLFiwIICIgK5PJqLn5fqXTd6un5yL19FykdPpuNTffTwshAAAABoyZ\nDJrZdWaWlHSnpP+U9Efufo2khKSPlji+0EokEqrdN1/Kv7bOSLX75iuRSAQWF5BKpdTR0aThzdYd\nHYsHKt8CAAAAhQwtcZyki9y9K3+mu2fM7PzShBV+sVhM6/7xEV1xyxXaOi3bLXTB/gVa+49rKR4D\nAAAAIPQKqSZ6tqTN7t6dm66RdJq7/1cZ4itIkIPOM84gwiaTySiRaFE6fbcGWwczamhoUTJ5N8co\nAABABSrV0BIpSWf0Z1tmFpP0grufMe5IJ1iQySAQRqnUZjU336+OjsWSpNraNsbABAAAqGClSgbT\n7t4wbN5P3f2D44ixJEgGgUPRag0AABAdpUoGvy2pTdJ9uVmfkrTE3ZeNJ8hSIBkEAAAAEGWlGnT+\nakl/Kul1STsk/TdJf1l8eAAAAACAsBizZXAyoGUQAAAAQJSNp2VwzKElzGyqpCslLZQ0tX++uzcX\nHSEAAAAAIBQK6Sb6b5JOkPRhSc9Keq+k7lIGBQAAAAAorYKGlnD3eH8FUTM7StJz7n52eUIcG91E\nAQAAAERZqQrIHMz9+zszO13SsZKOLzY4AAAAAEB4jPnMoKQ1ZjZT0l9LelzSdEl/U9KoAAAAAAAl\nddhk0Mxikva5+15JGyWdUpaoAAAAAAAlddhuou6ekfT5MsUCAAAAACiTQgrI/IOkNyT9u6QD/fPd\nfU9pQyscBWQAAAAARNl4CsgUkgz+coTZ7u6h6TJKMggAAAAgykqSDE4GJIMAAAAAomw8yeCY1UTN\n7LKR5rv7Q8X8IgAAAABAeBQytMSZea+nSjpP0ouSSAYBAAAAYJIqupuomc2QtM7dl5YmpOLRTRQA\nAABAlI2nm+hhh5YYxQFJJ49jOwAAAABASBTyzOD/kdTf7BaT9AFJ3yplUAAAAACA0ipkaInFeZO9\nkrrcfUdJoyoS3UQBAAAARFlJqolK2i7pV+7+du6XHGNmJ7n7q+OIEQAAAAAQAoU8M/iopEzedF9u\nHgAAAABgkiokGZzi7u/0T+ReH126kAAAAAAApVZIMrjbzC7onzCzCyW9UbqQAAAAAAClVkgBmfmS\nviFpTm7WDkmXufsvShxbwSggAwAAACDKxlNApuBB581suiS5+/5xxFZSJIMAAAAAoqwkg86b2d+b\n2Qx33+/u+81sppn97fjDBAAAAAAErZBnBv+Hu/+uf8Ld90r689KFBAAAAAAotUKSwXeZ2R/0T5jZ\nMZL+4DDrAwAAAABCrpBB578h6ftmtlaSSbpc0oOlDAoAAAAAUFoFFZAxs6WS/rskl7RP0gnu/lcl\njq1gFJABAAAAEGUlKSCT82tlE8GLJf2ZpC1FxgYAAAAACJFRk0EzqzOzW81si6S7JXUp25K4xN2/\nXOgvMLOlZvaymXWY2Q0jLL/UzNpzPz80sw8Wui0AAAAAYHxG7SZqZhlJ/1fSX7n7a7l5r7j7KQXv\n3CwmqUPSeZJ2Snpe0gp3fzlvnbMlbXH3N3PdUVe7+9mFbJu3D7qJAgAAAIisie4mepGkHkkbzexf\nzezPlC0gU4yzJHW6e5e7H5S0TtKF+Su4+yZ3fzM3uUnS3EK3BQAAAACMz6jJoLt/x91XSDpd0kZJ\nn5V0vJndZ2YfKnD/cyW9lje9Q4PJ3kiukvQf49wWAAAAAFCgMYeWcPcDkr4p6ZtmNlPZIjI3SHpq\nIgMxsyWSrpC0aDzbr169euB1U1OTmpqaJiQuAAAAAAibtrY2tbW1HdE+ChpaYtw7zz4PuNrdl+am\nvyDJ3f2OYet9UNJ6SUvdfVsx2+aW8cwgAAAAgMgq5dAS4/W8pFPNbJ6ZHS1phaTH81cwsxOVTQQ/\n0Z8IFrotAAAAAGB8xuwmeiTcvc/MPq1sl9KYpAfcfYuZrcou9jWS/kbScZLuNTOTdNDdzxpt21LG\nCwAAAABRUdJuouVCN1EAAAAAURbGbqIAAAAAgBAiGQQAAACACCIZBAAAAIAIIhkEAAAAgAgiGQQA\nAACACCIZBAAAAIAIIhkEAAAAgAgiGQQAAACACCIZBAAAAIAIIhkEAAAAgAgiGQQAAACACCIZBAAA\nAIAIIhkEAAAAgAgiGQQAAACACCIZBAAAAIAIIhkEAAAAgAgiGQQAAACACCIZBAAAAIAIIhkEAAAA\ngAgiGQQAAACACCIZBAAAAIAIIhkEAAAAgAgiGQQAAACACCIZBAAAAIAIIhkEAAAAgAgiGQQAAACA\nCCIZBAAAAIAIIhkEAAAAgAgiGQQAAACACCIZBAAAAIAIqphkMJPJBB0CAAAAAEwaFZMMJpYnlGpP\nBR0GAAAAAEwKFZMMphvSar6lmRZCAAAAAChAxSSDikkd1R1KpWgdBAAAAICxVE4yCAAAAAAoWOUk\ngxmprrtO8Xg86EgAAAAAIPSmBB3ARKlP1av1i62KxSonvwUAAACAUjF3DzqGI2Zm3tfXRyIIAAAA\nIJLMTO5uxWxTMdkTiSAAAAAAFI4MCgAAAAAiiGQQAAAAACKo5MmgmS01s5fNrMPMbhhh+QIz+5GZ\nvW1m1w9b9qqZtZtZysx+UupYAQAAACAqSlpN1Mxikr4s6TxJOyU9b2bfdfeX81b7raRrJS0bYRcZ\nSU3uvreUcQIAAABA1JS6ZfAsSZ3u3uXuByWtk3Rh/gru/oa7JyX1jrC9lSFGAAAAAIicUidacyW9\nlje9IzevUC7paTN73sw+OaGRAQAAAECEhX3Q+XPc/Vdm9h5lk8It7v7DkVZcvXr1wOumpiY1NTWV\nJ0IAAAAAKLO2tja1tbUd0T5KOui8mZ0tabW7L81Nf0GSu/sdI6x7q6Rud//nUfY16nIz81L+PwAA\nAAAgzMI46Pzzkk41s3lmdrSkFZIeP8z6A8GbWZWZTc+9nibpQ5J+XspgAQAAACAqStpN1N37zOzT\nkp5SNvF8wN23mNmq7GJfY2azJL0gqVpSxsyuk/QBSe+RtMHMPBfnN9z9qVLGCwAAAABRUdJuouVC\nN1EAAAAAURbGbqIAAAAAgBAiGQQAAACACCIZBAAAAIAIIhkEAAAAgAgiGQQAAACACCIZBAAAAIAI\nIhkEAAAAgAgiGQQAAACACCIZBAAAAIAIIhkEAAAAgAgiGQQAAACACCIZBAAAAIAIqphkMJPJBB0C\nAAAAAEwaFZMMJhItSqU2Bx0GAAAAAEwK5u5Bx3DEzMylPjU0tCiZvFuxWMXkuAAAAAAwJjOTu1tR\n21ROMuiqqlqvjRtPUiKRCDokAACAkjnppJPU1dUVdBgAAjBv3jy9+uqrh8wfTzI4ZaKCAgAAQHl0\ndXWpEm7oAyieWVH53mFVUH/KjOrqnlU8Hg86EAAAAAAIvYpJBuvrr1Nr6yqeFwQAAACAAlTMM4N9\nfX0kggAAIBJyzwYFHQaAAIz2+R/PM4MVkz2RCAIAAABA4cigAAAAUHG6uroUi8WUyWSCDgUhw7Ex\niGQQAACgQmQyGSWTSSWTyXFf6B7pPvbu3avly5dr+vTpOvnkk/XII48cdv277rpLs2fP1owZM3TV\nVVfp4MGD497XcBNZdXEixWIxvfLKK2X9nZPt2LjmmmtUXV2tmpoa1dTUaOrUqTr22GMHljc1NemY\nY45RTU2NqqurddpppxUVC8dG7veV7TcBAACgZFKpzUokWtTY2KXGxi4lEi1KpTaXfR+f+tSnNHXq\nVO3evVsPP/ywrrnmGm3ZsmXEdZ988kndeeed+sEPfqCuri5t27ZNt95667j2Vawgn7ksdyKSak8p\nsTyhxrsa1XhXoxLLE0q1p8q+j2L+nvfdd5+6u7u1b98+7du3TytXrtTFF188sNzMdO+992rfvn3q\n7u6esONCitaxIXef9D/Z/wYAAEA0DL/26evr84aGa13qc8lzP9l5fX19Be1zIvZx4MABP/roo/0X\nv/jFwLzLLrvMb7zxxhHXv/TSS/3mm28emH7mmWf8hBNOGNe++vr6/HOf+5z/4R/+oc+fP9+/8pWv\neCwWG4i9qanJb775Zj/nnHO8qqrKt23b5jt37vQLLrjAjzvuOK+trfWvfvWrA/tbvXq1/8Vf/IVf\ncsklXl1d7YlEwtvb2weWb9myxZuamnzGjBl++umn++OPPz6wrKmpyR944IGB6a9//eu+aNEid3dv\nbGx0M/Np06Z5dXW1f+tb3xr7jT0CfX193nBBg+sWuVbnfm6RN1zQUNyxcYT7KPbvmW///v1eXV3t\nzz333MC84e/xWPFX0rExWu6Tm19UHkXLIAAAwCSXSqXU0dGkoZ2+YuroWKxUqrDWm4nYR0dHh446\n6ijNnz9/YF59fb02bx65dXHz5s2qr68fsu5vfvMb7d27t+h9rVmzRk888YTa29v1wgsv6LHHHjtk\nnYcfflhf+9rX1N3drRNPPFErVqzQiSeeqF27dunRRx/VTTfdpLa2toH1H3/8cV1yySXau3evVq5c\nqWXLlqmvr0+9vb36yEc+oqVLl2r37t360pe+pI997GPq7Owc9b3pb/F59tlnJUk/+9nPtG/fviGt\nXaWQSqXUUd0x/M+qjuqO4o6NI9xHsX/PfOvXr9fxxx+vRYsWDZl/44036vjjj9e555478L6OhGNj\ndCSDAAAAmBD79+9XTU3NkHk1NTXq7u4edf3858Bqamrk7uru7i56X48++qhaWlo0Z84czZgxQzfe\neOMh61x++eV6//vfr1gspl27dulHP/qR7rjjDh111FGqr6/XVVddpYceemhg/UQioeXLl+td73qX\nrr/+ev3+97/Xpk2btGnTJh04cEA33HCDpkyZoiVLluj8888v6plGj9jQIMX+PfM99NBDuuyyy4bM\nu/POO/XKK6/o9ddf1yc/+Ul95CMf0S9/+csRt+fYGB3JIAAAwCQXj8dVV9cmKb+oR0Z1dc8qHo+X\nbR/Tp0/Xvn37hsx78803VV1dXdD6b775psxM1dXVRe9r586det/73jcwPW/evEPWyV++c+dOHXfc\ncaqqqhqyzeuvvz7i+mamuXPnaufOnYf8rpG2DYt4PK667rrhf1bVddcVd2wc4T6K/Xv22759u9ra\n2g5JBs8880xNmzZNRx11lC677DKdc845euKJJ0bcB8fG6EgGAQAAJrlYLKbW1lVqaGhRVdV6VVWt\nV339dWptXVXwWMwTsY+6ujr19vZq27ZtA/Pa29u1cOHCEddfuHCh2tvbB6bT6bRmzZqlmTNnFr2v\n2bNn67XXXhuY7urqOmSd/OIcc+bM0Z49e3TgwIGBedu3b9fcuXMHpvP35+7asWOH5syZozlz5mj7\n9u1D9p2/7bRp09TT0zOwbNeuXSPGXA6xWEytt7eqId2gqs4qVXVWqT5Vr9bbW4s7No5wH8X+Pfs9\n/PDDWrRokU466aTDrjfaQOwSx8ZhFfuQYRh/RAEZAAAQIaNd+/T19fkLL7zgL7zwQsGFPSZ6HytX\nrvRLL73UDxw44M8995zPmDHDX3rppRHX/d73vuezZ8/2l156yffs2eNNTU1+0003jWtf9913ny9c\nuNB37Njhe/bs8fPOO++QIiHDC440Njb6tdde62+//ba3t7f7rFmz/JlnnnH3bJGQo48+2jds2OC9\nvb3+T//0T37yySd7b2+vv/POOz5//ny/4447/ODBg/6DH/zAa2pqvKOjw93db775Zl+yZIn39PR4\nZ2en19bW+rnnnjvwe2fPnu1PP/100e/tkZhsx0a/BQsW+Ne//vUh8373u9/5k08+6W+//bb39vb6\nww8/7NOnT/fOzs4R91Fpx8Zon3+No4BM4IncRPyQDAIAgCgJ87XPnj17fNmyZT5t2jSfN2+er1u3\nbmDZ9u3bvbq62l977bWBeXfddZfPmjXLjz32WL/yyiv9nXfeKWhfw/X29vr111/v7373u/2UU07x\ne++9d8gF/5IlSw654H/99df9/PPP9+OOO85PPfVUX7NmzcCy1atX+8UXX+wrVqzw6upqP+OMMzyd\nTg8sf+mll3zx4sV+7LHH+sKFC/273/3uwLI33njDP/ShD3lNTY0vWrTIb7vttiEX/Pfff7/Pnj3b\nZ86c6Y8++mgxb++kVuyx8eMf/9inT5/u+/fvH7Kf3bt3+5lnnuk1NTU+c+ZM/5M/+RP//ve/P+rv\nrbRjYyKTQctuN7mZmVfC/wMAAKAQh+sSh4lx2223adu2bUOKhgBS8MfGaJ//3PyiBirkmUEAAAAA\niCCSQQAAAACIILqJAgAATDJ0EwWii26iAAAAAIAjQjIIAAAAABFEMggAAAAAETQl6AAAAABQnHnz\n5smsqEeDAFSIefPmTdi+KCADAAAAAJNcKAvImNlSM3vZzDrM7IYRli8wsx+Z2dtmdn0x2wJh19bW\nFnQIwIg4NhFmHJ8IK45NVJqSJoNmFpP0ZUkflrRQ0koze/+w1X4r6VpJ/ziObYFQ40sDYcWxiTDj\n+ERYcWyi0pS6ZfAsSZ3u3uXuByWtk3Rh/gru/oa7JyX1FrstAAAAAGB8Sp0MzpX0Wt70jty8Um8L\nAAAAADiMkhaQMbOPSvqwu/9lbvrjks5y98+MsO6tkrrd/Z/HsS3VYwAAAABEWrEFZEo9tMTrkk7M\nm35vbt6EblvsfxoAAAAAoq7U3USfl3Sqmc0zs6MlrZD0+GHWz0/qit0WAAAAAFCgkrYMunufmX1a\n0lPKJp4PuPsWM1uVXexrzGyWpBckVUvKmNl1kj7g7vtH2raU8QIAAABAVFTEoPMAAAAAgOKUfND5\nUmJQeoSZmb1qZu1mljKznwQdD6LLzB4ws1+b2U/z5s00s6fMbKuZPWlmxwYZI6JrlOPzVjPbYWYv\n5n6WBhkjosnM3mtmz5jZZjP7mZl9Jjef8ycCNcKxeW1uftHnzknbMpgblL5D0nmSdir7jOEKd385\n0MCAHDN7RVLC3fcGHQuizcwWSdov6SF3/2Bu3h2Sfuvud+Zups109y8EGSeiaZTjc0iFcSAIZnaC\npBPcPW1m0yUllR3z+gpx/kSADnNsXqIiz52TuWWQQekRdqbJ/RlDhXD3H0oaflPiQkkP5l4/ZizW\n3AAAA9BJREFUKGlZWYMCckY5PqWhReWAsnP3Xe6ezr3eL2mLstXtOX8iUKMcm/3jsRd17pzMF6oM\nSo+wc0lPm9nzZvbJoIMBhjne3X8tZb9UJB0fcDzAcJ82s7SZfY1ueAiamZ0kqUHSJkmzOH8iLPKO\nzf/KzSrq3DmZk0Eg7M5x9zMk/bmkv8p1hQLCanI+M4BKda+kU9y9QdIuSXQXRWBy3fAek3RdrhVm\n+PmS8ycCMcKxWfS5czIng0cyoD1Qcu7+q9y/uyVtULZrMxAWv84N7dP/7MFvAo4HGODuu32wqMFX\nJZ0ZZDyILjObouzF9r+5+3dzszl/InAjHZvjOXdO5mSQQekRWmZWlbtbIzObJulDkn4ebFSIONPQ\n5wgel3R57vX/kvTd4RsAZTTk+MxdYPe7SJw/EZxWSS+5+z158zh/IgwOOTbHc+6ctNVEpezQEpLu\n0eCg9P8QcEiAJMnMTla2NdAlTZH0DY5PBMXMvimpSdK7Jf1a0q2SviPpUUnvk9Ql6X+6+++CihHR\nNcrxuUTZZ2Aykl6VtKr/GS2gXMzsHEkbJf1M2e9zl3STpJ9I+pY4fyIghzk2L1WR585JnQwCAAAA\nAMZnMncTBQAAAACME8kgAAAAAEQQySAAAAAARBDJIAAAAABEEMkgAAAAAEQQySAAAAAARBDJIAAA\nOWbWZ2Yvmlkq9+/nJ3Df88zsZxO1PwAAjtSUoAMAACBEDrj7GSXcP4P7AgBCg5ZBAAAG2YgzzX5p\nZneY2U/NbJOZnZKbP8/Mvm9maTN72szem5t/vJl9Ozc/ZWZn53Y1xczWmNnPzex7ZvYHZfp/AQBw\nCJJBAAAGHTOsm+jFecv2uvsHJX1F0j25ef8iaa27N0j6Zm5akr4kqS03/wxJm3PzayX9i7ufLulN\nSR8t8f8HAIBRmTs9VgAAkCQz2+fuNSPM/6WkJe7+qplNkfQrd3+Pme2WdIK79+Xm73T3483sN5Lm\nuvvBvH3Mk/SUuy/ITX9e0hR3//uy/OcAABiGlkEAAArjo7wuxu/zXveJZ/cBAAEiGQQAYNCIzwzm\nXJL7d4WkH+de/6eklbnXH5f0XO71/5P0KUkys5iZ9bc2Hm7/AACUFXckAQAYNNXMXlQ2aXNJ33P3\nm3LLZppZu6S3NZgAfkbSWjP735J2S7oiN79F0hozu1JSr6RrJO0S1UQBACHCM4MAAIwh98xgwt33\nBB0LAAAThW6iAACMjTunAICKQ8sgAAAAAEQQLYMAAAAAEEEkgwAAAAAQQSSDAAAAABBBJIMAAAAA\nEEEkgwAAAAAQQf8fNgVSZ45WgQEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f410a776a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot train and validation accuracies of the two models\n",
    "\n",
    "train_accs = []\n",
    "val_accs = []\n",
    "for dropout in dropout_choices:\n",
    "  solver = solvers[dropout]\n",
    "  train_accs.append(solver.train_acc_history[-1])\n",
    "  val_accs.append(solver.val_acc_history[-1])\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].train_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Train accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "  \n",
    "plt.subplot(3, 1, 2)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].val_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Val accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question\n",
    "Explain what you see in this experiment. What does it suggest about dropout?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Answer\n",
    "Dropout can alleviate overfitting."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
